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This  study  effort  validates  the  appropriateness  of  em- 
ploying the  Delphi  technique  to  establish  measures  of 
effectiveness  for  support  type  functions. 

The  reason  some  processes  are  unmeasurable  and  the 
difficulties  of  measuring  outputs  are  discussed.  A model 
is  developed  for  determining  the  effectiveness  of  a security 
organization  and  a step-by-step  procedure  is  provided  for 
instituting  an  effectiveness  evaluation.  The  preparation  of 
a Delphi  questionnaire  for  establishing  goals  and  for 
weighting  the  goals  is  presented,  from  which  a panel  of 
security  experts  in  an  iterative  process  assigns  values  to 
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I . INTRODUCTION 

A.  BACKGROUND 

In  most  industrial  processes,  outputs  are  definable  and 
measurable.  A lathe  can  turn  down  2"  steel  rods  to  1.896" 

+ .005"  and  the  output  of  the  machine  is  quantifiable  and 
measurable.  And  so  it  is  with  other  product  type  operations. 
Output  may  be  in  inches,  pieces,  pounds  or  other  units  and 
its  attributes  are  easily  measurable.  Generally  speaking, 
the  outputs  of  most  processes  are  measurable. 

Because  output  is  a factor  in  computing  effectiveness, 
if  output  is  not  measurable,  how  then  is  effectiveness 
determined?  In  the  public  sector,  processes  abound  in  which 
output  is  vague  or  difficult  to  define.  Examples  appear  in 
such  activities  as  recreation,  public  health  and  national 
defense.  Quantifying  the  extent  of  physical  security 
provided  by  our  national  defense  effort  is  a baffling 
problem  for  the  experts  who  are  involved  in  its  study  on  a 
daily  basis.  Measures  such  as  flying  hours  used  or  extent 
of  border  patrolled  are  meaningless  when  related  to  the 
security  of  the  population.  An  aircraft  company  can  measure 
its  output  by  the  number  of  airplanes  leaving  its  plant. 

But  the  Air  Force  unit  that  uses  the  airplanes  and  has  as 
its  mission  the  task  of  protecting  the  coastal  perimeter 
from  enemy  intrusion  finds  that  just  defining  its  output  can 
be  a most  difficult  and  perhaps  impossible  job. 


9 


But  why  put  all  this  emphasis  on  output?  If  we  do  our 
job  is  it  not  enough?  The  answer  of  course  is  not  a simple 
yes  or  no.  3y  doing  our  job  we  gain  some  degree  of  satis- 
faction in  knowing  we  are  producing  something.  And  if  our 
product  measures  up  to  some  preselected  goal  we  are  also 
effective  in  our  job.  But,  if  the  output  is  vague  or 
unmeasurable,  and  no  goal  or  objective  has  been  established, 
we  may  not  only  be  unproductive  but  also  ineffective. 

Our  capitalistic  economy  established  profit  as  the 
primary  goal  for  industry.  Profit  is  that  which  enables 
business  concerns  to  survive.  In  fact,  in  a laissez-faire 
economy,  earned  profit  would  be  the  only  criteria  for 
determining  whether  or  not  a business  should  survive.  Maxi- 
mizing the  profit  while  holding  expenditures  constant  is  a 
very  viable  measure  of  effectiveness.  Government  activities 
however  are  seldom  engaged  in  profit  making  endeavors  so  a 
new  set  of  guidelines  is  required  for  effectivity  measures 
in  the  public  sector. 

A concept  analogous  to  profit,  one  that  is  increasingly 
used  by  government  when  profit  measures  are  nonexistent,  is 
benefit-cost  analysis.  Charles  J.  Stokes  [Ref.  1,  p.  184] 
cites  an  example  of  a highway  constructed  in  a developing 
nation  where  the  benefit  to  cost  ratio  was  computed  to  be 
8.3  to  1.  But  this  was  the  direct  benefit.  A factor  not 
considered  in  the  calculations  was  the  loss  of  the  agricul- 
tural land  with  a subsequent  reduction  in  food,  fibre  and 


fuel  production.  It  is  possible  that  more  productive  use 
could  and  should  have  been  made  of  the  capital  funds  used  to 
build  the  highway. 

Benefit-cost  analysis  can  suffer  from  the  same  malady  as 
effectivity  analysis  - the  difficulty  of  measuring  the 
benefit  or  output.  Measuring  the  benefit  is  usually  much 
more  difficult  than  measuring  the  cost.  Benefits  may  be 
difficult  to  express  in  monetary  terms  if  they  have  no 
market  price. 

3.  STATEMENT  OF  PROBLEM 

A model  has  been  developed  by  Lyden  and  Miller  for  eval- 
uating program  effectiveness  [Ref.  2,  p.  141]  that  requires 
a complete  description  and  measurement  of  each  variable  of  a 
program.  If  a variable  or  portion  of  a variable  cannot  be 
measured,  the  model  cannot  be  fully  applied.  In  many 
government  activities,  indicators  that  might  be  used  to 
judge  effectiveness  are  difficult  to  quantify.  A Naval  Air 
Station  provides  support  and  services  for  other  activities 
and  exists  as  a general  rule  only  as  a support  activity. 

The  major  portion  of  work  performed  by  Air  Station  personnel 
is  of  the  overhead  type,  although  some  direct  and  reimburs- 
able effort  is  accomplished. 

A Naval  Air  Station  Security  Department  performs  various 
functions  related  to  base  and  physical  security.  The  Air 
Station  Commanding  Officer  is  responsible  for  all  security 
within  his  command,  and  for  the  physical  security  of  all 
facilities  within  the  confines  of  the  Naval  installation. 
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Physical  security  is  that  part  of  an  overall  security 
program  which  is  concerned  with  the  physical  measures  de- 
signed to  prevent  unauthorized  access  to  equipment,  facil- 
ities, material  and  documents  and  to  safeguard  them  against 
espionage,  sabotage,  damage,  theft  or  other  covert  acts 
which  would  in  some  degree  lessen  the  ability  of  the  activity 
to  perform  its  mission  or  affect  overall  material  interests. 

The  difficulty  of  determining  the  effectiveness  of  this 
type  of  operation  is  apparent  from  the  above  description. 
Effectivity  measures  are  determined  by  describing  mission 
related  functions,  identifying  quantitative  indicators, 
assigning  objectives  or  goals  to  each  indicator  and  measur- 
ing the  degree  of  achievement  of  the  objectives. 

The  Security  operation  has  evaded  attempts  to  quantify 
mission  related  output  by  the  very  nature  of  its  work.  By 
skillful  application  of  cost  benefit  analysis  it  may  be 
possible  to  devise  a method  for  evaluating  alternatives  of 
choice,  but  no  treatment  has  evolved  for  applying  measures 
of  effectiveness  to  this  type  of  work.  This  paper  will 
attempt  to  devise  applicable  measures. 

C.  DIFFICULTIES  OF  MEASURING  SOME  PROCESSES,  OUTPUTS  OR 

ACCOMPLISHMENTS 

All  processes  have  value  in  that  some  money  is  paid  for 
them.  In  the  private  sector  of  our  economy,  values  are 
determined  in  markets,  and  value  is  equal  to  the  market 
price.  In  the  public  sector,  outputs  that  are  not  easily 
measurable  cannot  equate  to  a market  price  that  does  not 
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exist.  Here  the  problem  of  unmeasurables  forces  an  innova- 
tive method  for  evaluating  effectiveness  that  is  foreign  to 
private  enterprise.  A direct  example  is  a Security  Depart- 
ment in  a Naval  Air  Station.  How  do  you  measure  the  effec- 
tiveness of  base  security?  The  number  of  persons  apprehend- 
ed for  some  security  violation  might  give  a hint  as  to  the 
seriousness  of  the  problem.  But  how  deep  is  the  problem? 

What  percentage  of  the  total  violators  are  caught?  Just  how 
effective  are  the  security  operators? 

The  problem  might  be  viewed  from  the  opposite  perspec- 
tive. How  many  workers  have  never  been  cited  for  a viola- 
tion? Or,  what  percentage  of  our  assets  are  intact?  Would 
any  of  these  measures  be  a reflection  of  how  well  the 
security  department  is  performing?  Regardless  of  how  we 
view  the  situation,  measuring  the  effectiveness  of  a security 
organization  is  a complex  task. 

The  police  are  generally  assumed  accountable  for  in- 
creases in  all  types  of  crime.  It  is  questionable  whether 
they  should  be  thus  held  responsible  any  more  than  the  Fire 
Department  is  held  responsible  for  the  number  of  fires. 
Although  both  agencies  employ  preventive  strategies  and 
investigate  incidents  after  the  fact,  fire  departments  are 
rated  for  insurance  purposes  primarily  on  their  ability  to 
minimize  the  damage  due  to  fires  rather  than  on  the  number 
of  fires  [Ref.  3,  p.  454],  One  can  view  the  police  role 
similarly,  as  that  of  minimizing  the  harm  caused  by  crime  - 
or  caused  by  other  problems  handled  by  the  police.  There  is 
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a major  difference  between  developing  measures  of  police 
performance  and  developing  measures  to  determine  effective- 
ness of  programs  to  reduce  crime. 


D.  THE  SIGNIFICANCE  OF  EFFECTIVENESS  MEASURES 

An  organization's  mission  is  characterized  by  one  or 
more  objectives  which  represent  the  output  of  that  organiza- 
tion. A manager  is  given  certain  inputs,  or  resources  which 
he  uses  to  produce  outputs.  After  the  outputs  are  generated, 
performance  feedback  information  is  needed  to  indicate  how 
well  the  manager  utilized  his  resources  to  produce  the  out- 
puts. In  effectiveness  measures  we  are  concerned  with  a com- 
parison between  present  period  performance  and  stated 
objectives . 

E.  DEFINITIONS 

1 . Objectives 

A situation  or  condition  which  responsible  program 
personnel  consider  desirable  to  attain  [Ref.  2,  p.  145]. 

The  objective  should  contribute  to  attainment  of  the  mission. 
To  permit  subsequent  evaluation,  the  statement  of  an  objec- 
tive must  specify 

- The  nature  of  the  situation  or  condition  to  be 
attained. 

- The  quantity  or  amount  of  the  situation  or  condi- 
tion to  be  attained. 

- The  particular  group  of  people  or  portion  of  the 
environment  in  which  attainment  is  desired. 

- The  geographic  area  of  the  program. 

- The  time  at  or  by  which  the  desired  situation  or 
condition  is  intended  to  exist. 
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2.  Productivity  or  Output  Measures 


Productivity  is  broadly  defined  as  the  ratio  of  out- 
put per  unit  of  input  over  time.  Productivity  compares  the 
amount  of  resources  used  with  the  volume  of  products  or 
services  produced.  In  the  long  run,  productivity  is  improved 
by  innovation  or  changes  in  the  means  of  production.  In  the 
short  run,  productivity  is  improved  by  raising  the  efficien- 
cy with  which  resources  are  utilized  within  the  existing 
system  of  production  [Ref.  4]. 

The  productivity  concept  refers  to  an  accurately 
measured  total  physical  output  of  end-products  of  an  organi- 
zation, divided  by  the  measure  of  the  total  amount  of 
resources  used,  such  as  total  number  of  man-hours  worked  or 
total  cost  in  constant  prices.  In  order  to  determine  the 
changes  in  aggregate  outputs  and  factor  inputs,  and  thus 
productivity,  it  is  necessary  to  combine  unlike  types  of 
output  and  of  input  units  by  weights  that  indicate  their 
relative  importance  for  the  purpose  at  hand  [Sef.  5,  p.  8], 

Productivity  indices  are  not  precise  measures  of 
productiveness.  However,  one  of  the  principal  uses  of 
productivity  indices  is  in  the  area  of  manpower  forecasting 
for  both  short  and  long-range  planning.  Productivity 
measures  can  be  used  to  assess  manpower  utilization  and 
allocation  and  very  importantly,  it  can  be  used  to  assess 
the  effect  of  capital  equipment  expenditures  on  output. 

The  emphasis  here  as  in  much  of  the  private  and 
public  sector  has  been  in  increased  productivity  with  no 
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mention  of  effectivity.  The  literature  on  productivity  or 
efficiency  is  voluminous.  A plethora  of  studies  have  been 
undertaken  and  numerous  models  developed  for  determining 
productivity  indices  but  few  works  exist  on  effectiveness. 

There  is  an  awareness  that  has  been  growing  over  the 
last  few  years  that  there  is  a need  for  measures  of  effec- 
tiveness in  relationship  to  costs. 

Effectiveness  has  been  a concern  of  engineers  since 
1887  when  Arthur  M.  Wellington  wrote  his  treatise  on  the 
economic  theory  of  Railway  Location.  It  was  during  the 
1930's  that  the  concept  of  cost  benefit  analysis  into 
evaluation  of  public  works  was  introduced.  This  was  an 
approach  that  compared  the  benefits  translated  into  dollars 
with  the  equivalent  dollar  cost. 

During  the  1940 's  Eugene  Grant  brought  about  a wide- 
spread awareness  on  the  part  of  engineers  of  the  need  for 
economic  evaluation  of  engineering  projects.  Following 
World  War  II,  the  field  of  operations  research  provided  a 
greatly  expanded  viewpoint  of  economic  evaluation.  Tools 
such  as  linear  programming,  methods  of  optimization,  and  the 
like,  coupled  with  the  means  for  solution  afforded  by  the 
computer,  have  made  possible  more  meaningful  evaluations 
[Ref.  6,  p.  2,  3]. 

3 . Efficiency 

Efficiency  is  described  as  the  ratio  of  actual 
performance  numbers  to  standard  performance  numbers  usually 
expressed  as  a percentage. 
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ACTUAL  PERFORMANCE  (NUMBERS) 
STANDARD  PERFORMANCE  (NUMBERS) 


X 100  [Ref.  7,  p.  588] 


Ireson  and  Grant  in  their  Handbook  of  Industrial 

Engineering  and  Management,  (Ref.  8 ] in  speaking  of 

organization  efficiency  state: 

The  criterion  of  efficiency  requires  the  fulfil- 
ment of  the  personal  and  individual  objectives 
of  those  who  are  connected  with  the  enterprise. 

This  definition,  it  should  be  noted,  departs 
somewhat  from  general  business  usage.  "Effic- 
iency," used  in  this  sense,  rests  on  the  premise 
that  the  company  has  met  the  criterion  of 
effectiveness  and  can  fulfill  its  financial 
obligations . 

Deniston,  Rosenstock,  Welch  and  Getting  [Ref.  2, 

p.  143]  describe  efficiency  as: 

The  cost  in  resources  of  attaining  objectives. 

The  efficiency  of  a program  may  be  unrelated 
to  its  effectiveness,  adequacy,  and  appro- 
priateness . 

Efficiency  then  is  defined  in  different  ways  depend- 
ing upon  objectives.  For  a production  operation  it  is  a 
ratio  of  actual  to  standard  performance.  For  an  organization 
it  is  the  fulfillment  of  personal  and  individual  objectives, 
and  for  a program  it  is  the  cost  in  resources  of  attaining 
objectives . 

4 . Effectiveness 

Effectiveness  is  variously  described  as  (1)  the 
extent  to  which  preestablished  objectives  are  attained  as  a 
result  of  planned  activity  [Ref.  2,  p.  143],  or  (2)  a com- 
parison between  present  period  performance  and  stated 
objectives  [Ref.  6,  p.  3]  , or  (3)  the  desirable  effects  or 


benefits  gained  by  reason  of  the  expenditure  or  incurring  of 
a cost.  Effectiveness  also  connotes  some  measure  of  per- 
formance or  level  of  output  of  the  benefit  producing  system 
[Ref.  9,  p.  4]. 

Terms  Often  Used  to  imply  the  concept  of  effective- 
ness . 

Utility  - Favored  by  economists 

Productivity  - Results  oriented 

Worth  - Favored  by  engineers 

Merit  - Operations  research  carryover 

Benefit  - Favored  by  Bureau  of  the  Budget 

F.  SCOPE  OF  THIS  THESIS 

In  searching  the  literature  on  this  subject,  no  current, 
applicable  works  were  found  to  provide  guidance  on  measuring 
the  effectiveness  of  a Naval  Air  Station  security  force.  The 
existence  of  studies  in  cost  effectiveness,  organizational 
effectiveness  and  personal  effectiveness  all  strive  to 
improve  corporate  profit  as  a final  measure  of  effectivity. 

To  provide  a means  of  determining  the  effectiveness  of  a 
security  organization  a model  for  a measure  of  effectiveness 
was  developed  and  is  the  subject  of  this  thesis. 

To  determine  effectiveness,  specific  measures  of  accom- 
plishment for  preestablished  objectives  are  evaluated  in 
relation  to  goals.  A comparison  of  performance  to  stated 
objectives  will  provide  a basis  for  determination  of  effec- 
tiveness. The  mechanics  of  measurement  in  a profit  making 
organization  have  been  thoroughly  described  in  other  works. 
This  model  develops  a method  for  arriving  at  (1)  objectives, 
(2)  quantitative  indicators  and  (3)  a concensus  opinion 
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through  an  iterative  process  utilizing  the  Delphi  technique. 
The  Delphi  method  is  credited  with  providing  a method  for 
selecting  realistic  and  realizable  goals  and  assigning 
specific  weights  for  each  indicator. 
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II . DEVELOPMENT  OF  A MODEL  FOR  A MEASURE  OF  EFFECTIVENESS 

In  the  development  of  a model  for  effectiveness  measures, 
several  basic  considerations  are  predominant.  These  consid- 
erations may  be  listed  in  the  following  sequence: 

A.  Describe  organization  mission  requirements. 

B.  Determine  the  primary  elements  accomplishing 
the  mission. 

C.  Identify  quantitative  indicators. 

D.  Establish  quantitative  goals  or  objectives  for 
each  indicator. 

E.  Weight  each  indicator. 

F.  Measure  degree  of  achievement  of  each  goal. 

G.  Evaluate  effectiveness. 

A.  MISSION  REQUIREMENTS 

In  a military  establishment,  the  organization's  mission 
is  relatively  easy  to  determine.  A statement  of  mission  may 
be  obtained  from  the  organization  manual  or  directive  from 
higher  authority.  The  mission  statement  must  be  examined 
for  goal  directing  statements.  The  phrasing  of  the  mission 
will  provide  information  for  the  development  of  the  next 
step . 

3.  ELEMENTS  FOR  ACCOMPLISHING  MISSION 

Functions  performed  in  support  of  the  mission  may  be 
identified  from  the  mission  statement.  A determination  must 
be  made  of  whether  the  effort  will  lead  to  accomplishment  of 
the  ultimate  mission.  A program  may  encompass  several  sub- 
objectives in  order  to  achieve  the  ultimate  objective.  A 
careful  review  of  the  performance  of  the  organization's 
activities  will  provide  information  of  a primary  objective 
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or  subobjective  nature.  In  the  measurement  of  effectiveness, 
it  is  only  the  actions  that  lead  to  achievement  of  the 
primary  or  ultimate  goal  that  are  used  in  this  model. 

As  an  example,  the  security  department  registers 
vehicles  for  authorized  entry  on  the  base.  A portion  of  the 
mission  infers  that  unauthorized  entry  will  be  prevented. 

The  act  of  registering  the  vehicles,  while  a necessary  part 
of  overall  control,  will  not  in  itself  satisfy  the  mission 
requirement.  The  prevention  of  unauthorized  access  will  be 
accomplished  by  a gate  check  and  a display  of  valid  identi- 
fication. The  registering  of  vehicles  is  considered  to  be  a 
subobjective  and  need  be  evaluated  only  if  the  sum  of  the 
subob'jectives  will  be  used  in  the  determination  of  the 
ultimate  mission  objective.  Major  attention  to  objectives 
is  necessary  because  of  the  importance  of  distinguishing 
between  objectives  and  subobjectives. 

The  security  program  is  characterized  by  several  objec- 
tives which  represent  the  desired  end  result  of  the  security 
activities.  Each  objective  implies  one  or  more  subobjectives 
or  intermediate  objectives  which  must  be  accomplished  in 
order  that  the  security  program  objective  may  be  accomplished. 

To  perform  a meaningful  evaluation  it  is  necessary  first 
to  establish  the  specific  mission  that  the  program  is  to 
fulfill. 

In  a conference  with  the  Security  Department  Management 
Personnel,  requirements  were  identified  whose  fulfillment  is 
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essential  to  the  attainment  of  the  security  mission.  These 
requirements  are  in  consonance  with  the  department  mission 
s tatement . 

The  security  objectives  were  cuouped  into  three  major 
categories : 

1.  Base  admission 

2.  Failure  to  secure 

3.  Response  time 

In  describing  the  objectives,  emphasis  was  given  to  pro- 
viding those  that  are  measurable.  The  department  mission 
statement  specifies  that  unauthorized  access  to  equipment, 
facilities,  material  and  documents  will  be  prevented.  The 
base  admission  category  satisfies  this  requirement.  Cate- 
gory 2,  failure  to  secure,  was  selected  to  describe  that 
part  of  the  mission  that  requires  safeguarding  against 
espionage,  sabotage,  damage,  theft  or  other  covert  acts,  and 
protecting  equipment,  facilities,  material  and  documents. 

It  is  intended  that  this  objective  will  focus  on  the  number 
of  incidents  occurring. 

The  last  objective  is  response  time  which  will  provide 
an  indication  of  the  time  delay  between  receiving  a call 
for  assistance  and  the  actual  arrival  of  the  security 
personnel . 

C.  QUANTITATIVE  INDICATORS 

The  most  difficult  part  of  an  effectiveness  study  in  a 
government  service  organization  is  to  identify  meaningful 
quantitative  indicators.  This  step  requires  identification 
of  the  kind  of  evidence  needed  to  determine  that  an  objective 
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has  or  has  not  been  achieved.  The  validity  of  a measure  is 
the  extent  that  an  obtained  score  measures  the  characteris- 
tic that  it  is  intended  to  measure. 

The  use  of  indicators  to  assess  program  effectiveness 
has  been  sharply  criticized  by  Sheldon  and  Freeman  [Ref.  10, 

p.  80]. 

The  use  of  indicators  to  measure  outcomes  of 
programs  could  lead  to  the  most  egregious 
statistical  manipulation,  for  herein  lies  the 
arbitrary  selection  and  control  of  variables 
that  could  virtually  prescribe  results  without 
reference  to  important  determining  factors  and 
casual  interrelationships. 

The  criticism  of  the  use  of  indicators  points  out  the 
need  to  avoid  an  arbitrary  selection  and  to  perform  an 
unbiased  analysis  and  to  select  variables  that  are  meaning- 
ful and  contribute  to  the  accomplishment  of  the  mission.  In 
accord  with  this  reasoning,  each  of  the  primary  security 
functions  was  broken  down  into  elements  that  could  yield  a 
numerical  value.  The  total  value  of  the  elements  when 
properly  weighted  should  provide  an  indication  of  the 
effectivity  of  the  primary  function.  For  the  primary 
function  "Base  Admission",  three  quantitative  indicators 
were  selected.  These  are:  (1)  Unauthorized  people  in  the 

control  area;  (2)  Unauthorized  vehicles  on  the  base;  (3) 
Unauthorized  material  or  equipment  that  is  brought  aboard 
the  base  such  as  guns,  explosives,  drugs,  etc. 
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D.  ESTABLISHED  GOALS 


The  setting  of  goals  can  be  accomplished  in  a variety  of 
ways,  such  as  through  an  intuitive  process,  or  more  commonly 
by  a review  of  historical  trends.  The  method  recommended 
here  and  applied  to  the  Naval  Air  Station  Security  Depart- 
ment was  formulated  using  the  Delphi  technique  for  reaching 
a concensus  of  opinion."" 

Each  indicator  that  was  selected  was  evaluated  for  the 
extent  to  which  achievement  could  reasonably  be  obtained 
giving  consideration  to  existing  constraints.  A group  of 
security  experts  through  an  iterative  process  established 
achievable  goals  that  were  to  be  used  in  the  effectiveness 
determination . 

The  goals  will  provide  a basis  for  comparison  with 
actual  performance  to  yield  a measure  of  effectiveness. 

It  has  been  suggested  that  for  a security  function, 
nothing  less  than  1.00  reliability  is  acceptable.  Relia- 
bility as  used  in  this  context  is  defined  as  the  probability 
that  a security  violation  will  go  undetected  for  a given 
period  of  time.  As  an  example,  one  of  the  quantitative 
indicators  establishes  a goal  for  the  percentage  of  unauthor- 
ized people  in  the  control  area.  Precisely  what  is  meant  by 


See  Appendix  for  a description  of  the  questionnaires 
used  in  this  study. 
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stating  that  the  goal  for  this  indicator  is  0.2?  Generally, 
it  is  interpreted  to  mean  that  98  times  out  of  100,  un- 
authorized personnel  will  be  prevented  from  entering  or  be 
apprehended  in  the  control  area.  It  also  means  that  two 
times  out  of  100  unauthorized  personnel  will  be  undetected 
in  the  control  area. 

Security  like  safety,  is  one  of  the  most  difficult 
criteria  to  evaluate.  Western  society  places  an  almost 
infinite  value  on  human  life.  Yet,  if  some  risks  were  not 
acceptable,  space  missions  would  never  be  undertaken.  It  is 
recognized  that  the  risk  of  death  accompanies  many  accepted 
human  activities  and  that  these  risks  are  realized  and 
accepted.  The  risk  associated  with  air  travel  is  accepted 
for  the  greater  convenience  over  other  modes  of  transporta- 
tion. The  same  rationale  can  be  applied  to  security.  If 
some  risks  are  not  acceptable,  the  base  would  be  forced  to 
close.  Even  the  expenditure  of  huge  amounts  of  money,  and 
the  installation  of  the  most  advanced  technological  detec- 
tion systems  would  not  give  100%  assurance  that  a security 
violation  will  go  undetected. 

E.  ESTABLISHING  WEIGHTING  FACTORS 

The  relative  contribution  for  achievement  of  goals  can 
be  established  by  the  assignment  of  weights  to  each  indica- 
tor. A high  weight  indicates  that  more  importance  or  more 
emphasis  is  attached  to  that  element  than  to  one  with  a 
lower  weight.  The  weights  are  determined  in  a manner  similar 
to  that  for  which  values  were  assigned  to  the  goals. 
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The  group  of  security  experts  were  asked  to  evaluate 

each  indicator  and  the  quantitative  goal  established  for 

each  and  to  assign  a weight  that  would  provide  an  indication 

of  the  relative  contribution  of  each  one.  The  Delphi  method 

was  used  to  arrive  at  a concensus  opinion  for  the  weighting 

of  each  indicator.  To  illustrate  how  the  weights  are 

applied,  the  following  example  is  given: 

The  specific  goal  for  unauthorized  people  in 
the  control  area  is  2%,  and  the  weight  assigned 
is  5%.  (The  total  weight  distribution  to  all 
elements  is  100%) . If  a survey  revealed  94%  of 
the  people  in  the  control  area  were  authorized 
to  be  there,  the  percentage  of  goal  achieved  would 
be  94/98  or  96%.  The  weighted  program  element 
is  96%  x 5%  = 4.8%.  The  sum  of  all  the 
weighted  elements  will  provide  a total  program 
valuation  as  a percentage  of  a possible  100. 

F.  MEASURE  DEGREE  OF  ACHIEVEMENT  OF  EACH  GOAL 

Once  goals  have  been  established  it  is  necessary  to 
determine  how  well  the  organization  did  in  meeting  the  goal. 
The  measurement  may  be  accomplished  in  one  of  several  ways. 

1.  Survey  the  Population 

This  may  be  accomplished  by  taking  a random  sample, 
and  through  a statistical  analysis  a determination  can  be 
made . 

2 . 100%  Inspection 

While  not  very  practical  due  to  the  high  cost  and 
time  required,  such  a check  can  be  accomplished  by  inspect- 
ing every  vehicle  and  person  coming  aboard  the  station. 
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3 . Use  of  Planted  Personnel  or  Conditions 

This  method  would  require  the  selection  of  personnel 
to  attempt  an  unauthorized  entry.  The  number  apprehended 
would  give  an  indication  of  the  effectiveness  of  the  secur- 
ity force.  The  same  method  can  be  applied  to  classified 
material  or  open  safes.  By  deliberately  allowing  selected 
unsecured  classified  material  or  safes,  an  indication  of 
effectiveness  can  be  had  from  the  number  identified. 

G.  EVALUATE  EFFECTIVENESS 

The  evaluation  of  effectiveness  will  consider  the  degree 
of  achievement  of  each  goal,  the  weight  assignment  and  the 
overall  measure  of  effectiveness  for  the  organization.  Each 
indicator  should  be  reviewed  to  determine  if  added  emphasis 
is  required  in  any  area.  The  evaluation  phase  of  the 
program  is  most  important  in  arriving  at  a measure  of 
effectiveness . 
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GROUP  PROCESSES 


A.  GROUP  INTERACTION 

When  faced  with  a decision  that  isn't  clearly  determined 
by  very  good  information,  we  can  turn  to  a group,  a council, 
a committee,  a panel  or  a commission.  The  reason  for  using 
this  procedure  is  that  with  any  group  there  is  more  informa- 
tion collectively  in  the  heads  of  the  group  than  there  is  in 
the  head  of  a single  member  of  the  group.  Unfortunately, 
that  is  not  quite  enough,  because  with  any  group  there  is 
more  misinformation  than  there  is  in  any  one  head,  and  by 
convening  the  group,  you  may  be  putting  the  misinformation 
together  to  come  up  with  the  worst  judgement. 

The  standard  way  of  using  a group  is  to  convene  a face- 
to-face  discussion.  In  short,  issues  are  raised,  they  are 
hashed  over,  information  is  traded,  and  the  group  is  asked 
to  come  up  with  their  best  judgement  on  the  subject. 

The  traditional  way  of  pooling  individual  opinions  is  by 
face-to-face  discussion.  Norman  Dalkey  of  the  Rand  Corpora- 
tion described  some  difficulties  with  face-to-face  inter- 
action [Ref.  11,  p.  3]  as  follows: 

We've  conducted  experiments  at  Rand  where  we've  con- 
vened groups  and  asked  questions  for  which  we  knew 
the  answers,  but  no  one  in  the  group  really  knew  the 
answer.  When  the  face-to-face  discussion  technique 
was  used  the  answers  of  the  group  got  worse  after 
the  discussion.  We  polled  them  beforehand,  asking 
them  to  give  their  judgement  on  these  questions:  then 
we  let  them  talk  it  through  and  reformulate  their 
judgements.  More  often  than  not,  after  the  discus- 
sion the  judgements  were  worse  than  they  were  in  the 
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beginning.  The  major  reason  is  that  in  a group 
discussion  there  are  always  one  or  two  dominant 
individuals  who  take  over  the  discussion  and  do 
most  of  the  talking.  In  our  experiments  we 
found  that  you  can  almost  predict  the  direction 
in  which  the  group  opinion  is  going  to  shift  by 
just  measuring  the  amount  of  time  that  each 
individual  talks . 

From  the  above  it  can  be  concluded  that  group  opinion  is 
highly  influenced  by  the  person  who  talks  the  most,  and  there 
is  very  little  correlation  between  pressure  of  speech  and 
knowledge . 

A second  element  that  interferes  with  arriving  at  sound 
judgement  is  the  social  situation  that  prevails.  Much  of 
the  discussion  has  to  do  with  maintaining  prestige  and 
relative  position  in  the  group.  Individual  and  group  inter- 
ests that  are  not  related  to  the  problem  solving  purpose  of 
convening  the  group  consume  much  of  the  time  and  effort  of 
the  group. 

A final  consideration  is  group  pressure  for  conformity. 
The  group  itself  has  an  enormous  effect  and  creates  a strong 
pressure  on  the  more  timid  members  of  the  group  to  conform. 
Distortion  of  individual  judgement  can  occur  from  group 
pressure . 

B.  DELPHI  METHODOLOGY 

The  Delphi  is  a group  process  which  utilizes  written 
responses  as  opposed  to  bringing  individuals  together.  It 
lets  people  remain  anonymous  and  prevents  domination  by 
certain  individuals.  Delphi  avoids  the  difficulties  of 
face-to-face  discussion  and  at  the  same  time  maintains  the 
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obvious  advantage  of  the  group.  Delphi  is  essentially  a 
series  of  questionnaires.  The  first  asks  individuals  to 
respond  to  a broad  question.  Each  subsequent  questionnaire 
is  built  upon  responses  to  the  preceding  questionnaire.  The 
process  stops  when  concensus  has  been  approached  among  par- 
ticipants, or  when  sufficient  information  exchange  has  been 
obtained  [Ref.  12,  p.  83]. 

Delphi  was  originally  developed  by  the  Rand  Corporation 
in  the  late  1940's  to  obtain  the  most  reliable  consensus  of 
a group  of  experts.  Essentially,  the  Delphi  is  a series  of 
intensive  interrogations  of  individual  experts  concerning 
some  primary  question  interspersed  with  controlled  feedback. 
The  procedures  are  designed  to  avoid  direct  confrontation. 

Interaction  among  the  experts  is  accomplished  through  an 
intermediary  who  gathers  the  data  from  the  experts  and 
summarizes  it  along  with  the  expert's  answers  to  the  primary 
question.  This  mode  of  controlled  interaction  is  a deliber- 
ate attempt  to  avoid  the  disadvantages  associated  with  more 
conventional  uses  of  experts  such  as  is  in  round  table 
discussions  or  direct  confrontation  of  opposing  views.  The 
developers  of  the  Delphi  argue  the  procedures  are  more  con- 
ducive to  independent  thought  and  allow  more  gradual  formu- 
lation to  a considered  opinion  [Ref.  13]. 

Typically,  the  answer  to  the  primary  question  is  a 
numerical  quantity.  It  is  expected  that  the  individual 
experts'  estimates  will  tend  to  converge  as  the  experiment 
continues  even  if  the  estimates  expressed  initially  are 
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widely  divergent.  Delphi  involves  three  general  ideas: 
anonymity,  iteration  and  feedback,  and  statistical  group 
response . 
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1 . Anonymity 

A formal  questionnaire  is  the  usual  mode  of  communi- 
cating the  opinions  of  members  of  the  group  and  no  answer  is 
matched  to  an  individual.  Anonymity  is  a way  of  reducing 
the  effect  of  dominant  individuals.  Delphi  can  also  be  used 
to  aggregate  judgements  where  people  are  hostile  toward  one 
another,  or  where  individual  personality  styles  would  be 
distracting  in  a face-to-face  setting. 

2 . Iteration  and  Feedback 

The  exchange  of  information  or  feedback  is  very 
carefully  controlled  by  the  intermediary.  It  is  planned 
that  the  iteration  occurs  in  a series  of  stages  and  at  each 
stage  the  information  which  has  been  generated  by  the  group 
in  the  previous  stage  is  fed  back.  In  this  manner  the 
socializing  and  personal  interest  discussions  are  eliminated 
and  the  views  of  individual  experts  are  not  subjected  to 
criticism  in  face-to-face  confrontation. 

3 . Statistical  Group  Response 

Normally,  after  several  iterations,  the  group 
opinion  converges.  That  is,  the  individual  opinions  come 
closer  and  closer  together,  but  do  not  come  to  a single 
opinion.  There  is  always  some  distribution  of  answers,  even 
on  the  final  round,  and  rather  than  force  a single  opinion, 
a statistical  measure  is  taken.  The  median  or  the  mean  is 
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used  as  the  group  response.  Even  at  the  very  end,  the 
opinion  of  every  member  of  the  group  still  plays  a role  in 
determining  the  group  answer. 

In  experiments  conducted  by  Dalkey  [Ref.  11],  he 
found  that  contrary  to  the  results  of  the  face-to-face  type 
of  iteration,  as  the  exercise  proceeds  more  often  than  not 
the  opinions  of  the  group  become  more  accurate. 

C.  DELPHI  PARTICIPANTS 

The  selection  of  a panel  of  participants  is  accomplished 
from  persons  who  can  exercise  expert  judgement  in  the  area 
under  consideration.  The  term  "experts"  is  intended  to 
describe  persons  who  have  important  knowledge  or  experience 
to  share.  The  persons  selected  to  serve  as  the  Delphi  work 
group  should  possess  relevant  information  or  experience 
toward  which  the  Delphi  is  aimed. 

The  ideal  panel  would  be  composed  of  persons  who  feel 
personally  involved,  are  motivated  to  participate  in  the 
delphi  study  and  have  pertinent  information  to  share.  The 
size  of  the  panel  is  variable,  but  with  a homogeneous  group 
not  more  than  ten  participants  is  recommended.  This  study, 
performed  at  the  Naval  Air  Station  Point  Mugu  Security  De- 
partment, had  a panel  of  six  members,  each  of  which  was 
intimately  knowledgeable  in  some  phase  of  the  security 
operation . 
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IV.  DELPHI  APPLIED 


In  developing  measures  of  effectiveness  for  the  Naval 
Air  Station  Security  Department,  the  Delphi  technique  was 
used  to  arrive  at  quantitative  goals  for  various  indicators 
and  to  weight  each  indicator  for  relative  importance  in 
relation  to  the  whole. 

The  key  to  the  Delphi  process  is  the  questionnaire.  If 
the  respondents  do  not  understand  the  initial  broad  questions 
they  may  answer  inappropriately  or  become  frustrated  with 
the  questionnaire  and  lose  interest. 

A.  THE  DELPHI  QUESTIONNAIRE  FOR  ESTABLISHING  GOAL 

The  questions  submitted  to  the  group  experts  were  formu- 
lated to  provide  an  indication  of  the  security  performance. 

A study  of  the  department  mission  statement  was  accomplished 
to  extract  information  that  could  be  formulated  into  goal 
directing  effort  of  a primary  objective  nature. 

Six  questions  were  prepared.  The  questions  fulfill  the 
requirement  that  they  will  provide  an  indication  of  primary 
objective  effectiveness.  Three  questions  relate  to  base 
admission  and  one  each  to  unauthorized  material,  unsecured 
safes  and  reaction  time.  The  purpose  of  the  questionnaire 
was  to  evolve  a numerical  value  for  each  indicator  that 
could  be  used  as  the  goal  in  striving  for  high  effective- 
ness. The  questions  and  a discussion  of  each  follows: 
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Out  of  every  100  unauthorized  persons  who  might 
gain  access  to  the  installation  through  normal  entry 
what  is  the  minimum  number  that  should  be  detected  by 


tm  gate  guards? 

Entry  of  persons  to  the  installation  is  accomplished 
by  displaying  an  identification  badge  to  the  gate  guard. 

All  personnel,  military,  dependents,  civilian  employees  and 
visitors  are  issued  identification  badges  that  are  required 
to  be  worn  on  outer  clothing  when  in  a classified  area.  The 
color  of  the  badge  indicates  the  type  of  security  clearance 
held  by  the  individual. 

Passengers  in  vehicles  entering  the  installation  hold 
their  identification  out  for  inspection  by  the  gate  guard 
before  being  waved  through.  The  identification  badge  has 
the  bearer's  picture,  name,  activity  and  department  on  the 
face.  The  reverse  side  has  other  vital  statistics. 

This  question  recognizes  the  fallibility  of  attempt- 
ing to  maintain  100%  authorized  only  entry.  In  considering 
monetary  and  manpower  constraints  the  question  is,  how  many 
unauthorized  entries  are  we  willing  to  live  with?  In  the 
first  iteration  the  mean  was  95  or  stated  another  way,  5% 
unauthorized  entries  are  acceptable.  The  second  iteration 
had  minor  changes  in  the  wording.  In  place  of  asking  for 
"the  minimum  number  that  should  be  detected,"  the  wording 
was  changed  to  "minimum  acceptable  number  that  should  be 
detected."  This  change  was  made  for  clarity  purposes. 
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The  mean  value  on  the  second  iteration,  accepted  as 
the  group  concensus,  was  98  (see  Table  I).  The  goal  estab- 
lished for  this  indicator  was  2%  of  unauthorized  entries 
will  be  acceptable.  Anything  more  than  2%  would  be  an  in- 
dication of  lowered  effectiveness. 


2 . 5-23?  Admission 

Out  of  every  100  unauthorized  persons  who  might  try 
to  gain  access  to  the  installation  through  other  than  norma l 
entry  gates,  what  is  the  minimum  acceotable  number  that 
should  be  detected  by  the  security  forct? 

The  Naval  Air  Station,  Point  Mugu,  has  several  miles 
of  ocean  frontage.  Undetected  access  in  remote  areas  is 
feasible  under  certain  conditions.  Recognizing  this  fact, 
the  group  mean  for  this  question  was  44  for  the  first  itera- 
tion and  61  for  the  second.  Most  of  the  estimates  were 
raised  on  the  second  round  after  considered  judgement  and  a 
review  of  the  answers  to  the  first  set. 


3 . case  Admission 

Out  of  every  100  unauthorized  vehicles  that  might 
try  to  gain  access  to  the  installation  through  normal  entry 
gates,  what  is  the  minimum  acceptable  number  that  should  be 
detected  by  the  gate  guards ? 

A vehicle  is  given  authorization  to  be  driven  on  the 
installation  if  the  owner  maintains  at  least  a minimum 
value  insurance  coverage  and  the  vehicle  passes  a routine 
safety  inspection.  Evidence  of  authorization  is  demonstra- 
ted by  a base  decal  affixed  to  the  bumper,  or  on  ether 
exposed  area  if  other  than  a car  or  truck.  Temporary 
vehicle  passes  are  issued  by  the  visitor  control  desk  for 
official  visitors.  When  a vehicle  approaches  the  gate,  the 
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Table 


and  then  at 


security  guard  will  glance  at  the  vehicle  decal 
the  driver  and  passenger  identification  before  allowing 
admittance . 

Violations  can  occur  by  not  having  a vehicle  decal 
or  temporary  pass,  or  having  an  out-of-date  decal  that 
might  indicate  expiration  of  the  driver's  insurance.  The 
Delphi  group  indicated  92%  on  the  first  questionnaire  and 
98%  on  the  second.  The  high  numerical  value  is  an  indica- 
tion of  the  ease  of  detection  of  unauthorized  vehicles.  On 
the  first  round,  five  out  of  six  participants  assigned 
values  of  100%  to  this  item.  The  second  iteration  had  four 
out  of  six. 


Unauthorized  Material 


Out  of  every  100 


ises  where  an  individual  might  tru 


to  bring  unauthorized  material  aboard  the  base  (guns,  ex- 
plisives,  drugs,  eta.) , what  is  the  minimum  acceptable 
number  that  should  be  detected  by  the  security  force ? 

Detection  of  unauthorized  material  is  most  difficult 
when  no  search  is  initiated  of  incoming  persons  or  vehicles. 
When  a security  guard  suspects  unauthorized  material  is 
being  covertly  brought  aboard  he  will  institute  a search. 

The  mean  value  for  this  item  on  the  first  iteration  was 
25.  One  participant  assigned  a value  of  zero.  This  was 
changed  on  the  second  iteration  where  the  mean  rose  to  33. 

5 . Unsecured  Safes 

Out  of  every  100  cases  where  a classified  material 
storage  container  has  been  left  unsecured  by  the  appropriate 
custodian,  how  many  should  be  detected  by  the  ^security  force': 
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(1) 

Within 

4 

hours 

(2) 

W i th i n 

3 

hours 

. / 

Wi  thin 

12 

h o urs 

(4) 

w ithin 

24 

hours 

Most  buildings  are  secured  after  hours  and  no  entry 
is  made  until  the  start  of  the  next  work  day.  Headquarters 
buildings  are  continuously  manned.  Individual  offices  how- 
ever, are  locked  at  night  after  the  custodial  crew  completes 
its  work.  Unsecured  classified  material  safes  may  be  dis- 
covered by  the  custodian  or  the  security  guard  making  a 
routine  check. 

Results  of  the  first  questionnaire  were  incomplete  so 
clarifying  remarks  were  provided  for  the  second  one.  The 
group  mean  for  this  question  showed: 

(1)  Within  4 hours  62% 

(2)  Within  8 hours  86% 

(3)  Within  12  hours  100% 

6.  Reaction  Time 


When  a security  alarm  is  sounded 
installation,  what  is  the  maximum  accepta 
between  when  the  alarm  is  sounded  and  the 
reaches  the  scene  of  the  incident? 


nywhere  on  the 
le  time  (in  minutes ) 
security  force 


When  a security  alarm  is  sounded,  it  alerts  the  dis- 


patcher in  the  security  building.  The  dispatcher's  desk  is 
manned  on  a 24-hour  basis.  The  dispatcher  in  turn  notifies 
the  security  patrol  by  radio  communication  and  the  security 
patrol  proceeds  to  the  scene  of  the  incident. 

On  the  first  iteration,  the  Delphi  group  assigned  a 
mean  value  of  3.4  minutes  for  the  security  patrol  to  respond 
to  the  alarm.  The  second  iteration  had  a mean  of  4.6  minutes 


which  was  the  accepted  value  for  this  item. 
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1 

THE  DELPHI  QUESTIONNAIRE  FOR  ESTABLISHING  WEIGHTS 
To  provide  a department  effectiveness  measure,  it  is 
necessary  to  integrate  the  various  indicators  into  one  overall 
rating.  The  method  for  doing  this  is  to  weight  each  indica- 
tor so  that  a maximum  value  of  100%  will  be  obtained  if  the 
organization  meets  the  goal  established  for  each  indicator. 

Indicator  number  six  on  the  Delphi  questionnaire  is  cited  as 
an  example  of  how  this  is  done.  The  statement  is,  "When  a 
security  alarm  is  sounded  anywhere  on  the  installation,  the 
maximum  acceptable  time  between  when  the  alarm  is  sounded 
and  when  the  security  force  reaches  the  scene  of  the  incident 
is  4.6  minutes."  This  indicator  is  given  a weight  of  32%. 

If  it  is  determined  that  the  average  response  time  during 
the  reckoning  period  is  5.4  minutes,  then  88%  of  the  goal 
has  been  achieved.  The  weighted  score  for  this  indicator 
would  then  be  38%  x 32%  = 27%.  Each  of  the  indicators  would 
be  weighted  in  this  manner,  and  the  weighted  values  would  be 
summed  up  to  give  a department  effectiveness  index.  The 
weight  for  each  indicator  was  established  by  using  the  Delphi 
method  in  the  same  manner  that  the  goals  were  established. 

See  Table  II  for  summary  of  weighting  values. 

The  six  indicators  that  were  used  to  establish  goals 
were  again  incorporated  into  a questionnaire  to  gain  the 
group  concensus  of  weighting  values  to  assign.  The  questions 
and  a discussion  of  each  follows: 


L 


39 


X.  3 CL  3 2 A.  dmt  3 3 Z O n 

Out  of  every  100  unauthorized  persons  who  might  try 
to  gain  access  to  the  installation  through  norma l entru 
gates,  S3_  should  be  detected  by  the.  gate  guards. 

The  Delphi  group  gave  this  indicator  a weight  of  nine 
on  the  first  iteration,  and  again  nine  on  the  second.  The 
low  value  of  two  and  the  top  value  of  ten  were  both  increased 
on  the  second  round.  The  increase  was  insufficient  however, 
to  raise  the  mean  higher  than  nine. 

2 . 3 ass  Admission 

Out  of  every  100  unauthorized  persons  who  might  try 
to  gain  access  to  the  installation  through  other  than  normal 
entry  gates,  3l_  should  be  detected  by  the  security  force. 

This  item  was  assigned  a weight  of  nine  on  the  first 
iteration  and  eight  on  the  second.  The  Delphi  group  saw 
little  relative  difference  between  this  indicator  and  the 
first  one  which  was  given  a weight  of  nine.  The  scores  had 
very  little  variance  indicating  a general  concensus  at  an 
early  stage. 

3 . Base  Admission 

Out  of  every  100  unauthorized  vehicles  that  might 
try  to  gain  access  to  the  installation  through  normal  entry 
gates,  93_  should  be  detected  by  the  gate  guards. 

Here  again,  differences  between  the  first  and  second 
iteration  were  small.  The  mean  value  was  five  on  the  first 


round  and  seven  on  the  second.  The  goal  for  this  indicator 
was  established  at  98%,  almost  a completely  perfect  target, 
yet,  the  security  experts  relegated  this  item  to  the  lowest 
relative  standing  of  the  entire  group. 
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4 . ' >• zu : ~ ri  zed  lla  teria  l 

u t of  every  100  cases  where  an  individual  might  try 
to  bring  unauthorized  material  aboard  the  base,  fauns,  ex- 
plosives, drugs,  eta.),  3_3  should  be  detected  by  the  security 
force . 


Weighting  for  this  item  was  seven  on  the  first 
round,  but  was  increased  to  13  on  the  second.  Some  of  the 
lowest  values  were  placed  on  this  one  in  the  first  iteration. 
The  final  weighting  moved  the  item  to  a relative  standing  of 
third  from  the  top. 


5 . Unsecured  Safes 


Out  of  every 
storage  container  has 
custodian,  S_2_  should 
be  detected  within  3 


100  cases  where  a classified  material 
been  left  unsecured  by  the  appropria 
be  detected  within  4 hours,  3_6_  should 
hours,  100  should  be  detected  within 


te 


12  hours. 


This  item  received  the  highest  weight  by  the  Delphi 
group.  It  was  given  a value  of  38  on  the  first  iteration 
and  32  on  the  second.  Unsecured  classified  material  is 
considered  to  be  the  most  serious  security  infringement  of 
all  the  indicators.  In  discussing  this  item  with  the 
assistant  security  officer,  he  expressed  the  thought  that 
all  other  security  effort  was  more  or  less  in  support  of 
this  one. 

6 . Reaction  Time 

When  a security  alarm  is  sounded  anywhere  on  the 
installation,  the  maximum  acceptable  time  between  when  the 
alarm  is  sounded  and  when  the  security  force  reaches  the 
scene  of  the  incident  is  4 . 6 minutes . 

This  last  item  was  ranked  second  in  importance  by 
the  rating  group.  On  the  first  iteration  it  was  given  a 
weight  of  32  which  was  changed  to  31  on  the  second  round. 
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Reaction  time  is  considered  to  be  of  major  importance  to 
base  security  and  the  maximum  acceptable  time  of  4.6  minutes 
is  an  indication  of  the  effort  expended  in  reacting  to  an 
alarm. 

C.  INTEGRATING  THE  ELEMENTS 

Having  established  specific  objectives  and  having  each 
objective  weighted,  it  is  now  necessary  to  combine  the 
various  elements  into  a single  index  of  effectiveness.  To 
accomplish  this,  a measure  of  achievement  in  relation  to  the 
objective  will  be  required.  This  measure  may  be  acquired 
using  one  of  the  methods  described  earlier.  To  illustrate 
how  a single  index  is  derived,  a hypothetical  situation  is 
described . 

Table  III  lists  the  six  objectives  and  their  numerical 
goals  in  column  1.  The  weight  assigned  to  each  objective  is 
shown  in  column  2.  The  achievement  of  the  security  force 
toward  each  goal  is  in  column  3.  The  values  listed  in 
column  3 are  hypothetical  and  are  used  to  illustrate  the 
mechanics  involved  in  determining  tbe  effectiveness  index. 
Column  4 is  the  percentage  of  achievement  of  the  goal  and  is 
computed  by  dividing  the  percentage  achievement  in  column  3 
by  the  values  in  column  1.  The  weighted  valuation  shown  in 
column  5 is  derived  by  multiplying  the  percentage  of  goal  in 
column  4 by  the  weight  in  column  2. 

It  is  evident  from  Table  III  that  the  objectives  given  a 
low  weight  contribute  only  a small  part  to  the  effectiveness 
index.  Item  3 achieved  100%  of  the  goal,  but  because  of  a 
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weight  of  7%  contributed  only  7%  to  the  effectiveness  index. 
It  is  hypothetically  possible  to  score  an  effectiveness 
index  greater  than  100%,  if  the  degree  of  goal  achievement  is 
greater  than  100%  for  several  of  the  objectives.  In  such  a 
case  it  will  be  necessary  to  evaluate  the  numerical  goal  to 
determine  if  a revision  is  warranted. 

The  effectiveness  index  calculated  in  Table  III  is  81.7%. 
Once  the  index  is  determined,  a periodic  evaluation  is 
possible.  Now  an  objective  rating  can  be  applied  with  the 
confidence  that  the  measure  of  effectiveness  is  a reflection 
of  the  department  effort. 


A 


V.  CONCLUSION 

A.  COST  CONSIDERATIONS 

In  the  planning  and  development  of  security  effectiveness 
measures  it  was  necessary  to  estimate  the  resource  input  that 
would  be  applied.  Budgetary  and  manpower  limitations  make  it 
infeasible  to  consider  the  application  of  more  funds  or  man- 
power than  has  been  allocated  to  the  department.  The  model 
developed  here  assumes  no  change  in  resource  input,  but 
sensitivity  analysis  can  be  undertaken  to  determine  the 
extent  of  a change  in  effectiveness  by  increasing  or  decreas- 
ing funds  or  manpower. 

It  was  stated  earlier  that  100%  assurance  can  not  be 
given  that  a security  violation  will  go  undetected.  One  of 
the  effectiveness  indicators  relating  to  base  admission  states 
that  "out  of  every  100  unauthorized  persons  who  might  try  to 
gain  access  to  the  installation  through  other  than  normal 
entry  gates,  61  should  be  detected  by  the  security  force." 

We  could  ask,  what  would  be  the  increase  in  cost  to  raise  the 
figure  to  some  number  higher  than  61? 

The  problem  is  to  determine  whether  an  additional  amount 
of  effectiveness  is  worth  the  added  amount  of  cost.  There 
are  two  methods  for  analysis  of  this  problem  without  resort- 
ing to  a purely  intuitive  decision  [Ref.  14,  p.  72]. 

1.  Specify  a level  of  effectiveness  which  must  be  met, 
and  select  that  method  which  meets  this  level  at  lowest 
total  cost. 
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2.  Specify  a level  of  cost  which  must  not  be  exceeded, 
and  select  that  method  which  provides  the  highest  level  of 
effectiveness  without  exceeding  that  cost. 

Usually  there  is  no  one  level  of  performance  which  can  be 
stated  as  an  absolute  requirement.  For  example,  consider  the 
case  mentioned  above,  where  61  out  of  100  unauthorized 
entries  would  be  detected.  It  follows  that  the  more  funds 
spent  for  security,  the  more  protection  would  be  provided, 
though  not  necessarily  in  a one  to  one  ratio. 

Presenting  the  decision  maker  with  a comparison  of  level 
of  effectiveness  and  cost  by  using  the  method  above,  the 
optimum  level  can  be  chosen. 

The  resources  expended  to  achieve  a given  objective  can 
in  most  cases  be  put  in  money  terms;  hence  we  try  to  compare 
benefits  or  effectiveness  with  the  cost  incident  to  it. 

What  makes  the  problem  difficult  is  that  the  benefits  are 
far  more  difficult  to  place  on  a monetary  scale,  and  the 
benefits  are  usually  displaced  in  time  from  the  expenditure 
of  resources. 

B.  APPLICATION  OF  TECHNIQUE  TO  OTHER  FUNCTIONS 

When  faced  with  limitations  of  time  and  money,  a per- 
functory analysis  can  not  treat  all  the  considerations  that 
may  be  relevant.  In  dealing  with  intangibles,  judgement 
and  intuition  can  be  applied,  but  deficiencies  in  quantita- 
tive measurements  are  apparent.  In  dealing  with  difficult  to 
quantify  considerations,  it  is  the  systematic  and  direct  use 
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of  expert  judgement  that  provides  a framework  for  decision 
making . 

A single  individual  is  not  likely  to  provide  the  range  of 
expertise  necessary  to  deal  with  complex  problems.  It  is  in- 
evitable that  a variety  of  experts  be  consulted.  It  is  for 
this  reason,  and  to  avoid  the  psychological  drawbacks  of 
face-to-face  discussion  that  the  Delphi  method  was  adopted. 

The  model  described  herein  is  not  intended  to  be  restrict- 
ed to  a security  organization  to  the  exclusion  of  other 
functions.  Its  application  is  appropriate  to  recreation, 
administration,  personnel,  legal,  air  operations,  human 
resources  and  other  support  type  functions.  The  Delphi 
technique  provides  a bridge  to  where  scientific  analysis  and 
objectivity  are  employable.  It  is  a method  to  make  the 
difficult  to  quantify  considerations  easier 


to  manipulate. 


APPENDIX 


This  appendix  contains  the  questionnaire  and  the  results 
of  each  iteration  for  establishing  quantitative  goals  and  for 
weighting  each  goal. 


QUESTIONNAIRE  FOR  SECURITY  PLANNING 

This  questionnaire  is  the  first  in  a series  designed  to 
gain  information  related  to  managing  the  security  effort. 

This  information  will  assist  in  providing  insight  into  the 
most  important  areas  of  security  aboard  the  installation,  the 
allocation  of  resources  to  those  areas  of  importance,  and 
measuring  the  effectiveness  of  the  ongoing  security  program. 
Your  expertise  in  the  security  field  will  assist  in  develop- 
ing these  insights.  Responses  will  not  be  attributed  to  the 
individual,  but  composite  results  will  be  distributed  to  each 
participant.  Please  reply  carefully  and  thoughtfully,  from 
your  professional  point  of  view. 


1.  Below  is  a list  of  indicators  which  might  be  used  to  help 
measure  the  effectiveness  of  the  security  force.  For  each  in- 
dicator listed,  please  indicate  the  number  of  violations,  in 
a one  month  period,  that  you  would  consider  as  a minimum  ac- 
ceptable operating  target  for  the  security  force  under  current 
funding  and  manpower  levels. 


a.  BASE  ADMISSION.  Out  of  every  100  unauthor- 
ized persons  who  might  try  to  gain  access  to  the 
installation  through  normal  entry  gates,  what  is 
the  minimum  number  that  should  be  detected  by  the 
gate  guards? 

(Example  of  possible  answer  - 75) 


b.  BASE  ADMISSION.  Out  of  every  100  unauthor- 
ized persons  who  might  try  to  gain  access  to  the 
installation  through  other  than  normal  entry  gates, 
what  is  the  minimum  number  that  should  be  detected 
by  the  security  force? 
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QUESTIONNAIRE  FOR  SECURITY  PLANNING 


This  questionnaire  is  the  first  in  a series  designed  to 
gain  information  related  to  managing  the  security  effort. 

This  information  will  assist  in  providing  insight  into  the 
most  important  areas  of  security  aboard  the  installation,  the 
allocation  of  resources  to  those  areas  of  importance,  and 
measuring  the  effectiveness  of  the  ongoing  security  program 
Your  expertise  in  the  security  field  will  assist  in  develop- 
ing these  insights.  Responses  will  not  be  attributed  to  the 
individual,  but  composite  results  will  be  distributed  to  each 
participant.  Please  reply  carefully  and  thoughtfully,  from 
your  professional  point  of  view. 

1.  Below  is  a list  of  indicators  which  might  be  used  to  help 
measure  the  effectiveness  of  the  security  force.  For  each  in- 
dicator listed,  please  indicate  the  number  of  violations,  in 
a one  month  period,  that  you  would  consider  as  a minimum  ac- 
ceptable operating  target  for  the  security  force  under  current 
funding  and  manpower  levels. 


a.  BASE  ADMISSION.  Out  of  every  100  unauthor- 
ized persons  who  might  try  to  gain  access  to  the 
installation  through  normal  entry  gates,  what  is 
the  minimum  number  that  should  be  detected  by  the 
gate  guards? 

(Example  of  possible  answer  - 75) 


b.  BASE  ADMISSION.  Out  of  every  100  unauthor- 
ized persons  who  might  try  to  gain  access  to  the 
installation  through  other  than  normal  entry  gates, 
what  is  the  minimum  number  that  should  be  detected 
by  the  security  force?  
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c.  BASE  ADMISSION.  Out  of  every  100  unauthor- 
ized vehicles  that  might  try  to  gain  access  to  the 
installation  through  normal  entry  gates,  what  is 
the  minimum  number  that  should  be  detected  by  the 
gate  guards? 

d.  UNAUTHORIZED  MATERIAL.  Out  of  every  100 
cases  where  an  individual  might  try  to  bring  un- 
authorized material  aboard  the  base  (guns,  explo- 
sives, cameras,  drugs,  etc.) , how  many  should  be 
detected  by  the  security  force  as  a minimum? 

(Be  realistic) . 


e.  UNSECURED  SAFES.  Out  of  every  100  cases 
where  a classified  material  storage  container  has 
been  left  unsecured  by  the  appropriate  custodian, 
how  many  should  be  detected  by  the  security  force? 


(1) 

Within 

4 

hours? 

(2) 

Within 

3 

hours? 

(3) 

Within 

12 

hours? 

(4) 

Within 

24 

hours? 

f.  REACTION  TIME . When  a security  alarm  is 
sounded  anywhere  on  the  installation,  what  is 
the  minimum  acceptable  reaction  time  with  which 
the  security  force  should  respond? 


2.  What  additional  quantitative  indicators  can  you  think  of 
which  are  not  presently  included  in  the  above  list? 
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6 July  1976 


RESULTS  OF  THE  FIRST  QUESTIONNAIRE 
FOR  SECURITY  PLANNING 

1.  Six  responses  to  the  initial  questionnaire  were  received. 

2.  Results  of  the  first  four  questions  are: 

abed 
90  10  100  10 

100  10  50  0 

90  50  100  5 

95  90  100  10 

99  2 *100  23 

98  10£  100_  100 

Mean  *Rounded  to  100  by 

(average)  95  44  92  25  the  referee 


QUESTIONS 

a.  BASE  ADMISSION.  Out  of  every  100  unauthorized 
persons  who  might  try  to  gain  access  to  the  installation 
through  normal  entry  gates,  what  is  the  minimum  number  that 
should  be  detected  by  the  guards? 

b.  BASE  ADMISSION.  Out  of  every  100  unauthorized 
persons  who  might  try  to  gain  access  to  the  installation 
through  other  than  normal  entry  gates,  what  is  the  minimum 
number  that  should  be  detected  by  the  security  force? 

c.  3ASE  ADMISSION.  Out  of  every  100  unauthorized 
vehicles  that  might  try  to  gain  access  to  the  installation 
through  normal  entry  gates , what  is  the  minimum  number  that 
should  be  detected  by  the  gate  guards? 

d.  UNAUTHORIZED  MATERIAL.  Out  of  every  100  cases  where 
an  individual  might  try  to  bring  unauthorized  material 
aboard  the  base,  how  many  should  be  detected  by  the  security 
force  as  a minimum? 

3.  The  fifth  question  referred  to  unsecured  safes  and  asked: 
Out  of  every  100  cases  where  a classified  material  storage 
container  has  been  left  unsecured  by  the  appropriate  custo- 
dian, how  many  should  be  detected  by  the  security  force? 
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6 July  1976 


QUESTIONNAIRE  FOR  SECURITY  PLANNING 


This  is  a continuation  of  the  exercise  which  began 
9 June  1976.  The  results  are  attached.  Please  answer  the 
reworded  questions  that  are  provided  below.  If  your  new 
answer  lies  at  one  of  the  extreme  ends  of  the  list  of  previ- 
ous answers  that  were  given  for  the  same  question,  please 
state  briefly  the  reason  why. 

1.  BASE  ADMISSION.  Out  of  every  100  unauthor- 
ized persons  who  might  try  to  gain  access  to  the 
installation  through  normal  entry  gates,  what  is 
the  minimum  acceptable  number  that  should  be 

detected  by  the  gate  guard?  


2.  BASE  ADMISSION.  Out  of  every  100  unauthor- 
ized persons  who  might  try  to  gain  access  to  the 
installation  through  other  than  normal  entry  gates, 
what  is  the  minimum  acceptable  number  that  should 
be  detected  by  the  security  force? 


3.  BASE  ADMISSION.  Out  of  every  100  unauthor- 
ized vehicles  that  might  try  to  gain  access  to  the 
installation  through  normal  entry  gates,  what  is 
the  minimum  acceptable  number  that  should  be  detec- 
ted by  the  gate  guards? 


4.  UNAUTHORIZED  MATERIAL.  Out  of  every  100 
cases  where  an  individual  might  try  to  bring  un- 
authorized material  aboard  the  base  (guns,  ex- 
plosives, drugs,  etc.),  what  is  the  minimum 
acceptable  number  that  should  be  detected  by  the 
security  force? 


5.  UNSECURED  SAFES . Out  of  every  100  cases 
where  a classified  material  storage  container 
has  been  left  unsecured  by  the  appropriate  cus- 
todian, how  many  should  be  detected  by  the 
security  force? 


(1) 

Within 

4 

hours? 

(2) 

Within 

8 

hours? 

(3) 

Within 

12 

hours? 

(4) 

Within 

24 

hours? 
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Note:  Fill  in  each  blank  space.  Example  of  possible 

answers : 

(1)  10 

(2)  30 

(3)  75 

(4)  99 


6.  REACTION  TIME.  When  a security  alarm  is 
sounded  anywhere  on  the  installation,  what  is 
the  maximum  acceptable  time  (in  minutes)  between 
when  the  alarm  was  sounded  and  the  security 
force  reaches  the  scene  of  the  incident? 


12  July  1976 


QUESTIONNAIRE  FOR  SECURITY  PLANNING 


This  is  a continuation  of  the  exercise  for  security 
planning.  The  items  addressed  in  previous  questionnaires 
are  repeated  below,  but  this  time  values  are  given  for  each 
item.  These  values  are  medians  that  were  calculated  from  the 
group  answers  provided  in  the  last  questionnaire. 

You  are  now  asked  to  weight  each  item.  One  hundred  (100) 
points  are  to  be  distributed  to  the  various  items  based  on 
your  perception  of  the  relative  importance  of  that  item  within 
the  list.  Please  distribute  these  100  points  in  the  space 
provided,  so  that  the  sum  equals  100. 

1.  BASE  ADMISSION.  Out  of  every  100  unauthorized 

persons  who  might  try  to  gain  access  to  the 
installation  through  normal  entry  gates,  9_8 
should  be  detected  by  the  gate  guards. 


2.  BASE  ADMISSION.  Out  of  every  100  unauthorized 

persons  who  might  try  to  gain  access  to  the 
installation  through  other  than  normal  entry 
gates,  6_1  should  be  detected  by  the  security  force. 


3.  BASE  ADMISSION.  Out  of  every  100  unauthorized 

vehicles  that  might  try  to  gain  access  to  the 
installation  through  normal  entry  gates,  9£ 
should  be  detected  by  the  gate  guards . 


4.  UNAUTHORIZED  MATERIAL.  Out  of  every  100  cases 

where  an  individual  might  try  to  bring  unauthor- 
ized material  aboard  the  base,  (guns,  explosives, 
drugs,  etc.)  , 3_3  should  be  detected  by  the 
security  force. 


5.  UNSECURED  SAFES . Out  of  every  100  cases  where  a 

classified  material  storage  container  has  been 
left  unsecured  by  the  appropriate  custodian, 

62  should  be  detected  within  4 hours 

86  should  be  detected  within  8 hours 

100  should  be  detected  within  12  hours 


d 
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REACTION  TIME.  When  a security  alarm  is  sounded 
anywhere  on  the  installation,  the  maximum  accept- 
able time  (in  minutes)  between  when  the  alarm  was 
sounded  and  when  the  security  force  reaches  the 
scene  of  the  incident  is  4.6. 


TOTAL 


20  July  1976 


( 


RESULTS  OF  THE  QUESTIONNAIRE 
FOR  WEIGHTING  SECURITY  GOALS 


1.  Six  responses  to  the  questionnaire  were  received. 


Individual  and  average  weight  for 
given  in  the  following  table. 

Question  Numbers 

each 

question  are 

1 

2 

3 

4 

5 

6 

— 

— 

2 

2 

2 

1 

20 

15 

10 

10 

3 

2 

30 

25 

10 

10 

4 

5 

40 

30 

10 

10 

5 

10 

45 

30 

10 

10 

10 

10 

45 

30 

10 

10 

10 

14 

50 

60 

9 

9 

5 

7 

38 

32  Average 
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20  July  1976 


QUESTIONNAIRE  FOR  SECURITY  PLANNING 

This  is  a continuation  of  the  exercise  for  security 
planning.  The  results  of  the  previous  questionnaire  for 
weighting  the  different  items  are  attached. 

You  are  asked  to  reconsider  the  answers  you  have  previ- 
ously given,  and  to  again  place  weighting  values  on  each 
item.  The  same  procedure  is  to  be  used  as  before,  where  100 
points  are  to  be  distributed  to  the  various  items  based  on 
your  perception  of  the  relative  importance  of  the  items  with- 
in the  list. 


1.  BASE  ADMISSION.  Out  of  every  100  unauthorized 

persons  who  might  try  to  gain  access  to  the 
installation  through  normal  entry  gates,  9_£ 
should  be  detected  by  the  gate  guards. 


2.  3ASE  ADMISSION.  Out  of  every  100  unauthorized 

persons  who  might  try  to  gain  access  to  the 
installation  through  other  than  normal  entry 
gates,  6_1  should  be  detected  by  the  security  force. 


3.  BASE  ADMISSION.  Out  of  every  100  unauthorized 

vehicles  that  might  try  to  gain  access  to  the 
installation  through  normal  entry  gates,  9_8 
should  be  detected  by  the  gate  guards. 


4.  UNAUTHORIZED  MATERIAL.  Out  of  every  100  cases 

where  an  individual  might  try  to  bring  unauthor- 
ized material  aboard  the  base,  (guns,  explosives, 
drugs,  etc.)  , 3_3  should  be  detected  by  the 
security  force. 


5.  UNSECURED  SAFES . Out  of  every  100  cases  where  a 

classified  material  storage  container  has  been 
left  unsecured  by  the  appropriate  custodian, 

62  should  be  detected  within  4 hours 

86  should  be  detected  within  8 hours 

100  should  be  detected  wirhin  12  hours 


59 


REACTION  TIME.  When  a security  alarm  is  sounded 
anywhere  on  the  installation,  the  maximum  accept- 
able time  between  when  the  alarm  was  sounded  and 
when  the  security  force  reaches  the  scene  of  the 
incident  is  4.6  minutes. 


TOTAL 
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